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, Abstract. In the regression model with errors in variables, we observe n i.i.d. copies of (Y, Z) satisfying Y = fgo {X) + 

^ ' ^ and Z = X -\-e involving independent and unobserved random variables X,^,e plus a regression function fgo , known 

ly^ I up to a finite dimensional 6''^. The common densities of the Xi's and of the ^i's are unknown, whereas the distribution 

. of e is completely known. We aim at estimating the parameter 6° by using the observations {Yi,Zi), . . . , {Y„, Zn). 

We propose an estimation procedure based on the least square criterion SgO g{d) = ¥,go ^g[{{Y — fe{X))^w{X)] where 
w is a weight function to be chosen. We propose an estimator and derive an upper bound for its risk that depends 
on the smoothness of the errors density and on the smoothness properties of w{x) fg{x) . Furthermore, we give 
sufficient conditions that ensure that the parametric rate of convergence is achieved. We provide practical recipes for 
the choice of w in the case of nonlinear regression functions which are smooth on pieces allowing to gain in the order 
' of the rate of convergence, up to the parametric rate in some cases. We also consider extensions of the estimation 

\ procedure, in particular, when a choice of we depending on would be more appropriate. 

^ I Resume. Dans le modele de regression avec erreurs sur les variables, nous observons n v. a. i.i.d. de meme loi 

• • , que {Y,Z) satisfaisant aux relations Y = fgo{X) + ^ et Z — X -\- e, on les v. a. X, ^, e sont independantes, pas 

^ ' observees, et la fonction de regression fgo est connue a un parametre de dimension finie d" pres. Les densites de 

k>( I X et de ^ sont inconnues tandis que la loi de e est entierement connue. Nous estimons le parametre 6^" a partir 

j_j . des observations {Yi,Zi), . . . , {Y„,Z„). Nous proposons une procedure d'estimation basee sur le critere des moindres 

' Carres Sgo^g{9) — E,go^g[{{Y — fg{X))^w{X)], ou w est une fonction de poids a choisir. Nous definissons I'estimateur 

et calculous la borne superieure du risque de cet estimateur, qui depend de la regularite de la densite des erreurs Pe 
et de la regularite en x de w{x)fe{x). De plus, nous etablissons des conditions suffisantes pour que les estimateurs 
atteignent la vitesse parametrique. Nous decrivons des methodes pratiques pour le choix de x dans le cas des fonctions 
de regression non-lineaires qui sont regulieres par morceaux permettant de gagner des ordres de vitesse allant jusqu'a 
la vitesse parametrique dans certains cas. Nous considerons egalement des extensions de cette procedure d'estimation, 
en particulicr au cas oii un choix de wg dependant de 6 serait plus appropie. 
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1. Introduction 

We consider the regression model with errors in variables where one observes n independent and identically 
distributed (i.i.d.) copies of {Y,Z) satisfying 

(Y = fgaiX)+^, 

involving independent and unobserved random variables X,^,s, plus a regression function fgo known up 
to a finite dimensional parameter 6*°, belonging to the interior of a compact set of C M'*. The common 
densities of the Xi's and of the ^i's are unknown, with E(^) = 0, whereas the density of the errors e^'s is 
completely known. In this context, we aim at estimating the finite dimensional parameter 9'^ in the presence 
of a functional nuisance parameter g, the density of the X . 

Previous known results 

This model has been widely studied with first results written in the I950's (see, for instance, [28] or [20]). 
Most of the results deal with linear models where -yn-consistency, asymptotic normality, and efficiency 
have been studied. One can cite among the others Bickel and Ritov [3], Bickel et al. [2], Cheng and van 
Ness [7], van der Vaart [30, 31, 32], Murphy and van der Vaart [26]. The nonlinear models have been 
more recently considered starting with the case of repeated measurement data as in [12, 33, 34], under 
additional assumptions as in [13, 17, 21, 22, 25] or by simulation (see [5, 18, 19, 24]). To our knowledge, 
the first consistent estimator in nonlinear regression models with errors in variables, under nonparametric 
assumptions on the design density g has been proposed by Taupin [29] when the errors e are Gaussian and 
by Comte and Taupin [8] in the context of auto-regressive models with errors in variables for various types 
of errors density p^. In those papers, the estimation procedure is based on the estimation of the modified 
least square criterion E[{Y — 'E{fg{X)\Z))'^W{Z)] where the conditional expectation is estimated by using 
the observations Zi,. . . ,Zn and where is a compactly supported weight function. It is also stated that 
the rate of convergence, which does not have an explicit form, depends on the smoothness of the regression 
function as well as the smoothness of through the increase of the ratio {fgp^{z — ■))* {t)/pl{t) as t goes 
to infinity. The parametric rate of convergence is achieved in some specific examples, such as polynomial 
or exponential regression functions. The main drawback of this estimator is that besides its complexity, its 
rate of convergence does not have an explicit form. 

More recently. Hong and Tamer [16] propose a consistent estimator in the specific case where p* is of 
the form p^it) = 1/(1 + a^t^). Their estimation procedure strongly depends on this particular form of p* 
through the ratio l/p^ always appearing in errors in variables techniques. The extension of their method for 
other errors density p^ seems thus not obvious. 

Our results 

We propose here a new estimation procedure based on the least square criterion that is more general, explicit, 
natural, and tractable than the one proposed by Taupin [29]. Moreover, it often provides better results and 
it allows to provide sufficient conditions to achieve the parametric rate. 

The procedure is based on the estimation, using the observations {Yi,Zi) for i = l,...,n, of the least 
squares criterion Sgo_g := Ego g[(F — fg{X))'^w{X)] where w is a positive weight function to be chosen. 

In this context, we naturally define in Section 2 the estimator 




where Kn is a kernel such that its Fourier transform satisfies K*^{t) ^ K*{t)/p*{tCn), for K* compactly 
supported. In the literature, such a kernel A'„ is commonly known as the deconvolution kernel. In the sequel, 
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Cn is a sequence which tends to oo and p* denotes the Fourier transform, defined by p* (t) — J c'*^p(a;) dx 
of an arbitrary square integrable function p. 

We show in Section 3 that under classical identifiability, moment and smoothness assumptions, 6i is a 
consistent estimator of 9° with a rate of convergence depending on two factors, the smoothness of {wfe), 
and the smoothness of the errors density pe- This partly comes from the fact that this estimation proce- 
dure is based on the estimation of the linear integral functional 'EgO g{Y'w{X)fe{X)) and ¥.00 g{w{X) fg (X)) 
using the observations (Fi, Zi), . . . , (y„, that is, by recovering information on {Y,X) using the obser- 
vation {Y,Z). More precisely, it depends on the smoothness of w{x)dfe{x)/d9 and w{x)d{fg{x))/d9 and 
the smoothness of the errors density Ps{x), as a function of x through the behavior of {wdfg/d9)*{t)/pl{t), 
{wd{fg)/d9y {t)/pl{t) as function oft-^oo. From this construction, we derive sufficient conditions that 
ensure that 9i achieves the parametric rate of convergence. 

The rate of convergence of the proposed estimator is thoroughly studied for various smoothness properties 
of wfg, wfg as well as their derivatives in 9 as functions of x and for various errors' densities p^. It appears 
that as for the nonparametric estimation of the regression function in errors in variables models, the smoother 
is the noise density Pe the slower becomes the rate of convergence of estimators. Nevertheless, in examples 
we can considerably improve the smoothness of the functions fg and fg by multiplying them by a properly 
chosen weight function. This automatically improves the rates of convergence of our estimator. 

The conditions ensuring that the -y/ri-consistency is achieved are also deeply studied. The main idea is 
that the actual shape of the regression function matters less than its smoothness (compared to the noise 
smoothness) . Those conditions are illustrated through various examples of regression functions in Section 4. 
The point is that these conditions allow to achieve -y/n-consistency in setups that were not known before. 

This estimation procedure is used and extended to construct various other estimators. 

Firstly, under conditions ensuring that Kgo_g{Yw{X)fe(X)) and Kgo_g{w{X)fg{X)) are estimated at the 
parametric rate ^/n, we propose in Section 5 a second estimator 02 of 9*^ . In that case, the parametric rate 
of convergence for the estimation of 9^ can be achieved and the asymptotic normality of this estimator 
is stated. The first estimator, 9i, based on a deconvolution kernel is more general and applicable in all 
setups, but for specific regression functions, the use of a deconvolution kernel is not required. In those cases, 
the second estimator is more simple. Nevertheless, the conditions ensuring that KgO g{Yw{X)fg{X)) and 
Ego g(w{X) fg {X)) are estimated at the parametric rate ^/n are not always fulfilled. 

Secondly, when the variance cr'^ 2^ Var(^) is known, we propose in Section 6 another estimation criterion 
based on wg depending on 9. It allows to improve the rate of convergence of the estimators, by smoothing 
in a better manner wgfg. For a large class of regression functions, w not depending on 9 works well, for 
instance for polynomial, exponential, cosines regression functions as well as for regression functions fg of 
the form fg{x) ~ ip{9)f{x). But sometimes, the smoothing properties of w will be improved by taking w 
depending on 9, e.g., in the case where the regression function has to be smoothed at some point related to 
9. 

In this context, we extend our procedure to the estimation of the least squares criterion 

Sgo^g -.^EgaJiY - fg{X)fwgiX)]-al2EgoJweiX)], 

using the observations (Yi, Zi), . . . , {Yn, Zn), where wg is a positive weight function, depending on 9, to be 
chosen and where ct| 2 = Var(^) is now supposed to be known. Using this criterion and analogously to the 

construction of 9i , we propose to estimate 9° by 9i defined by 

1 " f 

9i = argmin5„,i(0) with Sn,i{9) = -Y, m - fe{x)f - a| 2\wg{x)C^K^{Cn{x - Z^)) dx. 

Under classical identifiability, moment and smoothness assumptions, the estimator 9i is a consistent estima- 
tor of 0° with a rate of convergence depending on the smoothness of d{wg{x) fg{x)) / d9 , d{wg{x)fg{x))/d9 
and on the smoothness of the errors density Pe{x), as functions of x. From this construction, we derive 
sufficient conditions that ensure that 9i achieves the parametric rate of convergence. 
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As for the first extension, we propose another estimator 62 of 6'^ based on SgO g achieving -^/n-consistency 
under constructive conditions ensuring that Ego_g{Y^wg{X)), Ego g{Ywg{X)fg{X)) and ¥.go g{wg{X) fg {X j) 
can be estimated at the parametric rate ^/n. 

Let us have a look to the properties of our estimator in the context studied by Hong and Tamer [16]. Our 
estimation procedure, more general, allows to recover -yn-consistency, in the case of noise densities satisfying 
pl{t) = c\t\~'^{l + 0(1)) as \t\ Qo with regression functions fg having derivatives in 9 up to order 3, twice 
continuously differentiable functions of x. 

The drawback of smoothing by multiplication is that for particular regression functions, we can obtain 
infinitely differentiable functions but not analytic ones. In such a case, and if the noise has an analytic 
density, the parametric rate of convergence can not be attained by our method. Nevertheless, the smoothing 
technique significantly improves the rate by using a clever choice of weight function w. 

The main question that remains open is: Is it possible to construct a y^-consistent estimator of 6^ for 
all regression functions and without any restriction on the smoothness of the errors density? 

The paper is organized as follows. Section 2 presents the estimation procedure and Section 3 gives the 
asymptotic properties of this estimator. Those asymptotic properties and practical recipes are illustrated 
through examples in Section 4. Section 5 presents another estimator and Section 6 gives extensions of the 
two former estimators, illustrated by examples in Section 7. 

The proofs can be found in Section 8 with technical lemmas presented in the Appendix. 

2. The estimation procedure 

Notations 

We denote by x_ the negative part of x, \\ ip W^— ^ ip'^{x)dx, \\ Lp |joo= sup^gg |(/3(a:;)|. In the same way, 
p-kq[z) = /p(z ~ x)q{x) dx denotes the convolution of two square integrable functions p and q. The variance 
Var(^) = E($2) ig denoted by cr| 2 and E(^-*) = cr|_4. For 6 e M"^, || {1^2= ELi ^fc ^nd 9^ is the transpose 
matrix of 9. 

From now, P, E, and Var denote the probability Pgo^g, the expected value Ego g and, respectively, the 
variance Var^o g, when the underlying and unknown true parameters are 9° and g. 

The starting point of the estimation procedure is to construct an estimator based on the observations 
[Yi, Zi) for i = 1, . . . , ?i, of the least square criterion 

~Sgo^g{9) = E[(y - fg{X)fw{X)], (2.1) 

where w is a positive weight function to be suitably chosen. 
This estimation procedure requires the following assumptions. 
Smoothness and moment assumptions 

(Ai) For any 9 in 6*, the function 9 1-^ fg admits continuous derivatives with respect to 9 up to the order 3. 
(A2) The quantities E[w'^{X){Y — fg{X))'^] and their derivatives with respect to 9 up to order 2 are finite. 

We denote by Q the set of densities g such that (A2) holds. We subsequently assume that there exist two 
constants C(/|o) and C{fga), depending only on 6*° through the functions fga and fgo, respectively, such 
that 

(A3) Slipg^g II ||2< C(/2o),SUPgge II fgog \\l< C{fgo). 

(A4) supggQ |w/e|, |w| and sup^gQ |w/|| belong to Li(M). 
Identifiability assumptions 

(III) The quantity Sgo^g{9) — ct|2E(w(X)) +E[{fgo{X) — fg{X))'^w{X)] admits one unique minimum at 
9 = 9°. 
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(II2) For all 6* e 6) the matrix S^^} (0) = d'^ Seo^g{e)/d0^ exists and the matrix 



2w{X) 



dfoiX) 



de 



fdfeiX) 



is positive definite. 



2.1. Construction of the estimator 

We denote by /y.x and fy.z the joint densities of {y,X) and respectively of {Y,Z) satisfying in this model, 



fY,x{y,x)^g{x)p^{y- fgo{x)) and fy^ziy, z) ^ fY,xiy, ■) * Pe{z). 
Write Sgo^g{e) as 

SgoJ9)=E[{Y-fg{X)fwiX)]^ f (y- fg{x)fwix)fY,xiy,x)dydx. 



(2.2) 



We consider that satisfies the following assumption. 

(Ni) The density belongs to L2(IR) and for all x e R,p*{x) ^ 0. 

The assumption (Ni) is quite usual in density deconvolution. 

According to (2.2) and under (Ni), we naturally propose to estimate Sga g{9) by 

Sn,m = - y / - fe{xyfw{x)Kn,cAx - Z,)dx = - V ((y, - fefw)*Kn,cAZ^), 



(2.3) 



where i^n.Cn^') — ^n^niCn') is a deconvolution kernel defined via its Fourier transform, such that 
/ Kn {x)dx = 1 and 



K*cAt) ^ K*it/C„) 
p*{t) p*{t) ' 



(2.4) 



with K* compactly supported satisfying |1 — K*{t) \ < l|t|>i. 
Using this empirical criterion, we propose to estimate 9*^ by 

§1 = a.TgmmSn,i{9). 



(2.5) 



3. Asymptotic properties 

3.1. General results for the risk of 9i 

We say that a function t/j e Li(M) satisfies (3.6) if for a sequence d and under previous notation we have 



min \\ih*(K^ —1)11 +n ^ min 

q=1.2 ^ ^" '"1 9=1.2 



Pe 



0(1). 



(3.6) 



Theorem 3.1. Let 9i =0i(C„) be defined by (2.5) under the assumptions (Hi), (112), (Ni), (Ai)~(A3) and 
(A4). 

(1) For all of the sequences C„ such that w, wfg and wfg and their first derivatives with respect to 9 satisfy 
(3.6), E(||0i — f^'^llla) = 0(1), as 71 —> 00 and9i = 9i{Cn) is a consistent estimator of 9^ . 
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(2) Assume, moreover, that for all 9 £ 0, w, few and fgW and their derivatives up to order 3 with respect 
to e satisfy (3.6). Then E(l| - W},) = 0{^l) with ^„ - (^2^^- - + 14^(0°)/", 



. , d, where 



B„,,(0) =min{BW and K,, (0) = mm{V;W (0), 

anc? /or (7 = 1,2 



djwfe) 
djwfe) 



' K* 



pi 



09, 



89, ) p* 



{K*n - 1) 



Remark 3.1. It is noteworthy that for any integrable ^, one can always find a sequence Cn such that (3.6) 
holds. 

Remark 3.2. The terms Bf^ ^ and Vnj/n are, respectively, the squared bias and variance terms with as in 
density deconvolution, bigger variance for smoother error density p^ and smaller bias for smoother (wfg). 

As we can sec, the rate of convergence for estimating 9'^ is given by both the smoothness of the errors 
density p^ and the smoothness of few; more precisely by the smoothness of d{fgw)/d9 and d{fgw)/d9, 
as functions of x. Those smoothness properties are described in both cases, by the asymptotic behavior of 
the Fourier transforms. The slower rates are obtained for the smoother errors density p^, for instance for 
Gaussian e's. Nevertheless, those rates are improved with a proper choice of w such that (wfe), (wfg) and 
their derivatives with respect to 9 are smooth enough. In this context, the parametric rate of convergence 
is achieved as soon as the derivatives with respect to 9, of (wfg) and (wfg) as functions of x are smoother 
than the errors density p^ . 

3.2. Consequence: sufficient conditions to obtain the parametric rate of convergence 

We say that the conditions (Ci)-(C3) hold if there exists a weight function w such that for all 9 E0, 

(Ci) the functions (wfe) and (wfg) belong to Li(R), and the functions supgW* /pt, supg{fgw)* /pi, 
sviPgifgw)* /pI belong to Li(M) nL2(R); 

(C2) the functions supeg0(^^^)*/p* and supgeei^^^f^)* /P*e belong to Li(]R) n L2(M); 

(C3) the functions and belong to Li(]R) n L2(M). 

Theorem 3.2. Consider model (1.1) under the assumptions (Ai)-(A4), (IIi), (II2), (Ni) and the conditions 
(Ci)-(C3). Consider C„ such that for all 9 G 0, fgw, fgW and their derivatives up to order 3 satisfy 
(3.6). Then 9i defined by (2.5) is a ^Jn- consistent estimator of 9'-' which satisfies moreover that y/n{9i — 



9^) A/'(0, El), with El that equals 

n — >oo 



E 



2w{X) 



dfg{X)\(dfe{X) 



89 



89 



e=e« 



2w{X) 



dfe{X)\(8fg{X) 
89 



e=9'> 



where Eg^i is given by 

■fd{fiw-2Yfgw) 



(27t)"^E 



89 



(u) ^-^ 



du 



8{f^w - 2Yfew) 



89 



^—iuZ 



(u) 



p*{u) 



Au 
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3.3. Resulting rates for general smoothness classes 

We now precise the asymptotic properties of 6i when the decrease of these Fourier transforms are quantified 
through the following assumptions. 

(N2) There exist positive constants C_{ps) , C (p^) , P , p,C(, and uq such that 
C{pe) < \p;{u)\\u\''eMf3Hn <Cipe) for all \u\>uo. 

If /9 > in (N2), then the noise is called exponential noise or super smooth noise. And if p = in (N2), then 
by convention /? = 0, a > 1 and the noise is called polynomial noise or ordinary smooth. 

The smoothness properties of functions involving the regression function are also given by the asymptotic 
behavior of the Fourier transforms described as follows. 

(Ri) A function / satisfies (Ri) if / belongs to Li(]R)nL2(M) and if there exist a,b,r, and uo>0 such that 
Lif) < |/*(u)||u|''exp(6|it|'-) <L(/) <oo for aU |m| >uo. 

If r = in (Ri) then by convention 6 = and the function / is called ordinary smooth. If r > 0, the function 
/ is called super smooth. 

Corollary 3.1. Under the assumptions of Theorem 3.1, assume thatp^ satisfies (N2) and that for all 9 Cz O, 
(few) , ifgw) and their derivatives with respect to 0j, j = 1, . . . ,d up to order 3, satisfy (Ri ) . 

(1) For all the sequences C„ such that 

(^(2a-2a+i-p) exp{-2feC; + 2/3C^}/n = o(l) as n ^ +00, (3.7) 

E(||^^l — 6'*'||^2) = 0(<^^) = 0(1) as 00 and9i=0i{Cn) is a consistent estimator of 9*^ . 

(2) Moreover, (p^ is given by Table 1, according to values of parameters a,b,r,a,P and p. 



4. Examples and methodological advice (1) 

In this section, we propose a deep study of the asymptotic properties of the estimator through various 
examples of regression functions. We show that for many regression functions the practitioner may encounter 

Table 1 



Rates of convergence ipf-^ of 9i 










p = in (N2) 
ordinary smooth 


p>0 in (N2) 
super smooth 


wf go (Ri) 

b = r = 
Sobolev 


a<« + l/2 „-(2a-l)/(2«) 

a>Q + l/2 


(logn)-(2"-l)/p 


(Ri)r>0 




r<p (logn)'*('^-'-'P)exp{-26(i^)'-/''} 


C°° 




(logn)-4(a.-,p) + 2<.6/(/3-) 



u — ^, u, ct -r i/ ^ 

b = /3,a>a + l/2 n"! 
b>/3 n-i 



r > p n ^ 



Where A(a,r,p) = (-2a + 1 - r + (1 - r)-)/p. 
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there are a few simple smoothing weight functions to choose so that the rates improve significantly, up to 
parametric rate in many cases. This new procedure allows to achieve the parametric rate of convergence 
in lot of examples and especially in examples where the previously known estimator proposed in [29] does 
not. In all of these examples, the noise distribution is arbitrary, as far as it satisfies (Ni) and (N2) with 
p < 2. The two first examples simply show that this new and more general estimation procedure allows to 
recover previous known results in simple cases. The other examples provide new results that underlie the 
improvement due to this method. 

Example 1 (Polynomial regression function). Let fg be of the form fg{x) = X]fc=i ^fe^'^ and sat- 
isfying (N2) with p <2. Assume that E(y^) < 00 and that V,{Z'^p) < 00. Let K be such K*(t) = l|t|<i and 
/ei 1/7(2;) = cxp{—a;^/(4/?)}. Then conditions (Ci)-(C3) are satisfied. Consequently, the estimator 9 1 is a 
^/n-consistent and asymptotically Gaussian estimator of 0^ . 

Remark ^.1. In this example, one can also choose w= 1, provided that the kernel K has finite absolute 
moments of order p and satisfies J K{u) du = 0, for r ~ 1, . . . ,p. With these choices of w and K , 61 
remains a ^/n- consistent and asymptotically Gaussian estimator of 0^ . 

The -^/n-consistency as well as the asymptotic normality was already achieved with different estimators, 
in the linear case (see, e.g., [2, 3, 26, 31, 32]). In polynomial case, other y^-consistent estimators already 
exist, without proving the asymptotic normality (see, e.g. [29] and [8]) or in the polynomial functional errors 
in variables model, with fixed and not random Xi's (see, e.g., [14, 15] or [6]). It is noteworthy that our 
new estimation procedure, quite more simple and natural than the one proposed in [29], also provides the 
•\/n-consistency in this simple case. 

Example 2 (Exponential regression function) . Let fg be of the form fe{x) = exp(6'a;) and p^ satisfying 
(N2) with p<2. Assume that E{Y^) < 00 and that E[exp(20°Z)] < 00. Let K be such K*{t) = l|t|<i and let 
w{x) — cxp{— a;^/(4/3)}. Then the conditions (Ci)-(C3) are satisfied and the estimator 61 is a ^/n- consistent 
and asymptotically Gaussian estimator of 9^ . Once again, this new estimation procedure allows to achieve 
the ^Jn- consistency and the asymptotic normality in a simple example where a other ^Jn- consistent estimator 
is already known (see [29]). 

Example 3 (Cosines regression function). Let fg be of the form fgix) —^'^^i9jCos{jx) and p^ sat- 
isfying (N2) with p<2. Let K be such K*{t) =l|t|<x and let w{x) = exp{— a;^/(4/3)}. Then the conditions 
(Ci)-(C3) are satisfied and the estimator 9i is a ^/n- consistent and asymptotically Gaussian estimator of 
9". 

Remark ^.2. This example has already been considered in [29] and [8], but the ^Jn- consistency as well as 
the asymptotic normality is new in this context. More precisely, the estimator constructed in [29] has a rate 
of convergence of order exp(-\/Togn)/n for Gaussian errors. 

Example 4 (Cauchy regression function 1). Consider model (1.1) with fg{x) =9/{l-\~x'^) satisfying 
(Ri) with a = 0,b = 1/2 and r — 1 and p^ satisfying (N2) with p <2. Let K be such K*(t) — l|t|<i and let 
w{x) = (1 + a;^)''exp{— x^/(4/3)}. With our choice of w, the functions fgw, fgW and their derivatives in 9 
up to order 3 satisfy (Ri) with p < r ^2 or p = r ~ 2 and b> [3. Consequently, the conditions (Ci)-(C3) 
are satisfied and the estimator 9i is a ^/n- consistent and asymptotically Gaussian estimator of 9^ . 

This simple example underlies the importance of the smoothing weight function w in the construction 
of 9i. Indeed, for a Gaussian noise £, without a smoothing function w in front of the regression function, 
Theorem 3.1 predicts (as for the estimator in [29]) a rate of convergence of order exp(— y^Togn) instead of 
the parametric rate of convergence. 
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Example 5 (Laplace regression function). Consider model (1.1) with fe{x) — 9f{x) and f{x) = 
exp(— |a;|/2). The Fourier transform of f and hence of fg is slowly decaying, like as \u\ oo. The 

estimator 6i with w = 1 would not provide \fn consistent estimator for smoother noise densities (as soon as 
\pt{'U')\ <o(|u|~^) with \u\ —^oo). A closer look tells us that fg and its derivative in 9 is C°° except at one 
point x = Q. Therefore, a proper choice of w can smooth out at and make wfg, wfg and their derivatives 
in 9 infinitely differentiable functions in x. With such a choice of the weight function w, the estimator 9i 
attains the parametric rate of convergence for a large set of noise densities, such that p^ satisfies (N2) for 
some < p <1. Even if p>l and the noise is smoother than that (e.g., Gaussian for p = 2), the rate of 9i 
is much faster when using our choice of w then it would be for w = 1 or when using the estimator proposed 
in [29]. 

Let us be more precise on the suitable choice of w and define 

'^°-^(")^""p(' (^-«)4-^r )^'"-^'^"^' (4.1) 

where —00 < a < 6 < 00 are fixed and R> 0. Following Lepski and Levit [23] and Fedoryuk [11], p. 346, 
Theorem 7.3, the Fourier transform of this function is such that 

\KA^)\ ^ cexp(-C|w|^/(^+i)) as |w| ^ 00 

and c, C > are constants. Then take w like S'o.ioo or if'_ioo,o or their sum (for a i? > large enough). 

Another way to smooth without restraining to compact support is the following. Let w{x) = exp(— l/|x|^^) 
a weight function which smoothes at as i? > is large. 

We can vary the coeflacient in the exponential in the expression of w and check that fgw, fgW and their 
derivatives up to order 3 satisfy (Ri) with the same r = R/{R-\- 1) closer to 1 as i? becomes large and 6 > 0. 

If the noise satisfies (N2) with < p < 1, then we find R large enough such that r = R/{R + 1) > p, 
and thus the conditions (Ci)-(C3) are satisfied. Consequently, the estimator 9i is a y^-consistent and 
asymptotically Gaussian estimator of 9^ . 

If P > 1, for this choice of w, then 

Ell^i - = 0(l)(logn)(i-2a-r)/p^^p|_26(logn/(2/3))"/''}. 

Note that for the same regression functions with f{x) — exp(|x|) we can multiply the previous weight 
function w by cxp(— 4|a;|) or by cxp(— x^) in order to solve integrability problems without changing the 
previous conclusions. 

Example 6 (Irregular regression function). Consider model (1.1) with fg{x) = 6'l[_i (x) and p^ sat- 
isfying (N2). Let K be such that K*(t) = l|t|<i and take w = ^-i.i for a R> defined by (4.I ). 

If p — in (N2), then the conditions (Ci)-(C3) are satisfied. Consequently, the estimator 9i is a ^pn- 
consistent and asymptotically Caussian estimator of 9^ . 

If p > 0, then the best rate for estimating 9^ is obtained by choosing w = If'-i.i with R > sufficiently 
large such that wfg and wfg satisfy (Ri) with < r = R/ (i? + 1) < 1 as close to 1 as needed. 

It follows that if < p < 1, then we can find w = I'-\^\ (with R large enough) satisfying (Ri) with 
r = R/{R + 1) > p, and hence the conditions (C4)-(C7) as well as conditions (Ci)-(C3) are satisfied. Thus, 
9i is a ^/n- consistent and asymptotically Gaussian estimator of 9'^ . Whereas, if p> 1, for a suitably chosen 
w, then 

Ell^i =0(l)(logn)(i-2a-r)/p^^p|_25(iog„/(2/3))"/''}. 

Example 7 (Polygonal regression function) . Consider model (1.1) with fg{x) = 6*0 + 0ix + 6'2(a; — a)+ + 
9^\x — b\'^ and p^ satisfying (N2). Let K be such K*(t) = l|t|<i. This regression function is C°° except at 
points a and b where it is not differentiable. For R> 0, let w{x) ~ ^a-ioo,aix) +^a,b{x) + !^'6,6+ioo(2;); with W 
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defined in (4-1). The idea is that we can truncate the regression function (say on the interval [a— 100, 6+ 100], 
or larger) in order to smooth out at points a, b and the end points of the support of w. 

If the noise satisfies (N2) with < p < 1, then take R large enough such that r ~ R/{R+ 1) > p, and thus 
the conditions (Ci)~(C3) are satisfied. Consequently, the estimator 9i is a ^/n- consistent and asymptotically 
Gaussian estimator of 0'^ . 

If p>l in (N2), according to Table 1 

Ell^i =0(l)(logn)(i-2a-r-)/Pp^p|_26(logn/(2/3))'^/''}. 
Comments on the examples 5, 6 and 7 

In those three examples, 9i achieves the ^/n-raie of convergence provided that p^ is ordinary smooth or super 
smooth with an exponent p < 1. But, few wih satisfy (Ri) with r at most such that r < 1 and it seems, 
therefore, impossible to have {wgfg)*/p* in Li(]R) if the e^'s are Gaussian. For this regression functions, if the 
ei's are Gaussian, the least square criterion can not be estimated with the parametric rate of convergence, 
and hence could probably not provide a y^-consistent estimator of 0° in this context. Nevertheless, even 
in cases where the parametric rate of convergence seems not achievable by such an estimator, the resulting 
rate of the risk of 0i is clearly infinitely faster than the logarithmic rate we could have without a proper of 
w or by using the estimator proposed by Taupin [29]. 



5. Second estimation procedure and comments on the conditions ensuring the 
-yn-consistency 

5.1. Construction and study of the risk of a second estimator 

We now propose another estimator of 0*^, based on sufficient conditions allowing to construct directly a 
i/n-consistent estimator of SgO g defined in (2.1), based on (Yi, Zi), i = 1, ... ,n. 

We say that the conditions (C4)~(C7) hold if there exists a weight function w and there exist functions 
^e,e,j, j = I7 2, 3 not depending on g, such that for all 6 E O and for all g 



(C4) / y w{x)fY,x{y,x)dydx = J y $0^eAz)fY,z{y,z)dydz, 

yfe{x)w{x)fY,x (y, x) dy dx = y" ySo^eA^)fY,z dy dz 



and J w{x)fg{x)g{x)dx = J <I'g,i.^i{z)h{z) dz; 

(C5) For J = 1, 2, 3, E[sup,ge I I] < 0°; 

(Cg) For j = 1,2,3 and for &l\ 6 e 0,E[\^^§f^{Z)\^] < 00; 

(C7) For j = 1,2,3 and for all OeO, E[\^^^I^{Z)\] < 00. 

Note that ^61,6,3 exists as soon as the chosen weight function w is smoother than p^ in the way that w* /p* 
belongs to Li(K). Furthermore, (l>e,e,3 = ^e,3 does not depend on 6. We refer to Section 5.2 for details on 
how to construct such functions ^e,e,j- 

Under (C4)-(C7), we propose to estimate SgO g{9) by 

1 " 

SnAS) = -y^[{Y,^^e,e.3iZ^) ~ 2Y,^g ^,,2{Z^) + ^6 .e.l{Z^)], (5.2) 
n ^ — ^ 



n 

i=l 



and hence 0" is estimated by 

6i2 = argminS'„,2(f). (5.3) 
See 
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Remark 5.1. The main difficulty for finding such functions ^e^e,j ^^es in the constraint that we expect 
that they do not depend on the unknown density g. If we relax this constraint, obviously there are a lot of 
solutions. 

Theorem 5.1. Consider model (1.1) under the assumptions (Ai)-(A4), (IIi). (II2), (Ni) and the conditions 
(C4)-(C7). Then 62, defined by (5.3) is a y/n- consistent estimator of 6'^ which satisfies moreover that y/n{02 — 

0^) ^ A/'(0,S2), with S2 that equals 



E 



2w{X) 



dfg{X)\(dfe{X) 



de )\ 09 

where Eg, 2 equals 



6=00 



-'0,2 



E 



2w{X) 



dfe{X)\(dfg{X) 



89 



36 



e=0" 



E 



89 



6=90 



8{3g^,^l{Z)-2Y3g,,,2{Z)) 



89 



Remark 5.2. In the Example 2, one can also choose w=l and use that 

This implies that if we denote by 

, , 0X0(26*^) , ~ , , exp(0Z) 

^g,eAZ) = „, M '^0,e.2{Z)- ^ ' 



E[cxp(26'e)] 



E[cxp(fe) 



then E,,[^(,,,,i(Z)] =E3[/2(X)] and^g.^h^^g^,,2{Z)\=W.g«^^'^ fg{X)\. 
Consequently, Sn,2 satisfies 



n 

~S^,2{9)^-Y. 



Y, 



I " 

-YW^ ^2Y,i>e^,^2{Zi)+^g,eAZ, 

II ^ — ^ 



E[exp(fe) 

In this case ^2 is also a ^/n- consistent and asymptotically Caussian estimator of 9'^ . 



(5.4) 



Remark 5.3. In the Example 3, one can also choose w = 1 and use that E[cxp(ijX)] = E[exp(ijZ)]/E[exp(ije)]. 
This implies that if we denote by 



^e,eAZ) = \\l+Y^ 



exp(2ijZ) exp(— 2ijZ) 
exp(i(j + fc)Z) exp{-i{j -\- k)Z) exp{i{j - k)Z) exp(i(-j + fc)Z) 



ptU + k) 



P*A-{j + k)) 



P*eU - fc) 



P*ei-j + k) 



and 



^e,e,2iZ) = T; 



cxp(ijZ) cxp(-ijZ) 



2L PtU) Pti-J) 
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thenE[Sg^^^i{Z)]^E[f^{X)] and E[Y3e,eAZ)] fe{X)]. Consequently, Sn,2 satisfies 

1 " 

Snai^) = -VK^ - 2Y,^e,e.2{Z^)+^e,e.l{Z^)]. (5.5) 

n ^ — ' 

In this case 62 with w = 1 is again a ^Jn- consistent and asymptotically Gaussian estimator of 9'^ . In the 
same way, 9i with w = l is again a y/n- consistent and asymptotically Gaussian estimator of 9'^ . 

Remark 5.4- Consider the Example ^. // we choose w as for 9i, then the conditions (C4)-(C7) are fulfilled 
and 92 is a ^/n- consistent and asymptotically Gaussian estimator of 9^ . Consequently, in this example, 
the conditions (5.1) given in [9] and [10] are not satisfied when our conditions (C4)-(C7) hold. Hence our 
estimation procedure provides a ^Jn- consistent estimator when the estimation procedure in [10] and [9] fails. 

5.2. Comments on the conditions ensuring the y/n- consistency 

Comment 1. Let us briefly compare the conditions (Ci)-(C3) to the conditions (C4)-(C7). It is noteworthy 
that the conditions (C4)-(C7) are more general. For instance condition (Ci) implies (C4), with ^e.ej de- 
fined by <Pg ^ ^ = {w fg)* / pI , 5 2 = i'^fo)* /Pe ^''^-d '^e e 3~ /Pe- This comes from the following equalities 
E[^e,,a(Z)]=E[(w;/|)(X)], 

E[Y^^e,eAZ)] = E[f^go{X)^e,eAZ)] + al^E[^e,eAZ)] 

= {feo9,^e,e,3*Pe) +C^l2{9,^e,e,3*Pe) 

= if^og^w) +al2{g,w) = E[Y^w{X)] 

and 

E[Y3e,eAZ)] = {fg„g,Se,s,2*Pe) = {27r)-' {ifg^g)* ,{3g,, ^2)* pI) 

= (fgogJgw)=E[Y{wfg){X)]. 

But in the condition (C4), few, fgW and w are not necessarily in Li(M). Consequently, under (Ci)-(C3), 
the conditions (Ci)-(C4) hold and, by denoting ^0,eS = {^fg)* Ip%, ^9,e.2 = (wfg)* /pi and 'I'e,e,3 = w* /pi, 
we get that So,i =^0,2 with Y.q i and 2 defined in Theorems 3.2 and 5.1. Nevertheless, the conditions 
(Ci)-(C3) are more tractable and constructive conditions. 

Comment 2. The conditions (C4)-(C7) have to he related to the conditions given in [9, 10] and [1] in the 
context of the functional errors in variables model with Xi , . . . , Xn not random. In both mentionned papers, 
in order to construct ^/n- consistent estimator, they assume that there exist two functions <j)i and (f>2 such 
that 

E(0i(x + e,0)) = /e(a;) and E(02(2: + e, 0)) - (5.1) 

Clearly our conditions (C4)-(C7) are less restrictive than the condition (5.1), and hence they allow to achieve 
the parametric rate of convergence for various type of regression functions, by using the possibility of the 
choice of the weight function wg . 

For instance, let us reconsider Example 4 where fg{x) = 9/{\ + x"^) and Pe is the Gaussian density. Then 
the conditions given in [9] and [10] are not fulfilled. Whereas E[fg{X)w{X)] and E[fg(^X)w{X)] can be 
estimated with the parametric rate of convergence, by taking w{x) = (1 + x"^)^ exp{—x'^ /{4f3)). It follows 
from the fact that condition (C4)-(C7) are fulfilled in this special example (see previous comment for the 
construction of auxiliary functions (pg^i^j, j = 1 to 3 appearing in the condition). Nevertheless, such weight 
function are not always available, and, therefore, those conditions (C4)-(C7) are not always fulfilled. 
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6. Extensions of previous estimation procedures 

In the estimation procedure previously presented, the weight function is used in order to make wfo, wfg 
integrable and also in order to get smooth wfg, wfg and derivatives in 6. In a large class of regression 
functions, the weight function can smooth these functions without depending on 9. Sometimes, the smoothing 
properties and hence the rate of convergence are improved by making w to depend on 6. It appears, in 
particular, when the points where we need to smooth are related to 9. 

The second estimation procedure uses an estimator based on the observations (Yi, Zi), i = 1, . . . , n, of the 
least square contrast 

SgoJ9) = E[((y - fe{X)f - alMX)], (6.1) 

where wg is a positive weight function to be suitably chosen. This criterion, which requires the knowledge 
of Var(^), actually writes K[{{fga{X) — fg{X))'^wg{X)] and it is minimized in 9 = 9^ for any positive weight 
function wg. Subsequently, we consider the following assumptions. 
Identifiability and moment assumptions 

(12 1) The variance cr| 2 = Var(^i) is known. 

(122) The quantity SgO g{9) = E[wg{X){Y - fg^X))"^] - a1.^E{wg{X)) admits one unique minimum at 9 = 9°. 

(123) For £dl9e0 the matrix 5*^0^ (9) = Sgo^g{9)/d9^ exists and 



2wgo{X) 



dfeiX) 



d9 



e=e° 



(dfg{X) 



V 89 



0=00 



is positive definite. 



(A5) The quantities K[wg{X){Y — fg{X))'^] and their derivatives up to order 2 with respect to 9 are finite. 
According to (2.2), Sgo^g{9) is naturally estimated by the empirical criterion 
, n 

Sn,m^-Yl m ~ f0{x)f - 4.2]M^)Kn,cACn{x ~ Z,))dx, (6.2) 
i=l 

where Kn^c^i') = CnKn{Cn-) is a deconvolution kernel satisfying (2.4). Using this empirical criterion, under 
(Ni), we propose to estimate 0° by 

^1 =argmin5'„,i(^')- (6.3) 
0e0 



Theorem 6.1. Let 9i = 0i(C„) be defined by (6.3) under the assumptions (I2i)-(I23), (Ni), (Ai)-(A4) and 
(As) with w replaced by wg. 

(1) Then for all of the sequences C„ such that for all 9 E0, fgwg and fgWg satisfy (3.6), E(||0i(C„) — 
^''11^2) =0(1), as 00 and 0i(C„) is a consistent estimator of 9^ . 

(2) Assume moreover that for all 9 G 0, fgWg and fgWg and their derivatives up to order 3 with respect 
to 9 satisfy (3.6). Then E(||^i - 9^%) = 0{^l) with ipn given ipn = \\{^u,j)\\t^ with ipl^^ = Bl j{9°) + 
'^n,j{9'^) /n, j ~ 1, . . . ,d, where 



B„,,(0) =min{<l (0),i35(0)} and K,,(^) = min{K!^.(0), yi5.(^)}, 



B. 



[9] 
"J 



86, 



(^£„ - 1) 



/ 8{fgWg )\* 

V 89, 



8{flwg) 

89^ 
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and 



Pe 



80, 



Pe 



Remark 6. 1 . The Remark 3. 2 is still valid. 

6.1. Consequence: a sufficient condition to obtain the parametric rate of convergence with 6i 

We say that the conditions (C8)-(Cio) hold if there exists a weight function wg such that for aU 8 E 0, 

(Cg) the functions {wgfg),{wg) and (wgfg) belong to Li(M) and the functions {wg)*/pl, 
supgifgwg)* /p*,supg{f^wg)* /p* bclong to Li(M) nL2(M); 

(Cg) the functions snpg^ei^)* //e^^^Peeei^^rl* /fe and supege(^%^) Vp^ belong to Li(M)nL2(] 
(Cio) the functions /ptA^^^f^T /P*e and {^^i§pl)*/p* belong to Li(R) nL2(R). 

Theorem 6.2. Consider model (1.1) under the assumptions (I2i)~(I23), (Ni), (Ai)-(A4) for w replaced by 
wg and under (A5), (C8)-(Cio). Then 9i defined by (6.3) is a ^/n- consistent estimator of 9° which satisfies 
moreover that y/n{di —0") ^ J\f{0,T,i), with Si that equals 



E 



2wgiX) 



dfg{X)\(dfg{X) 



89 



89 



g=g'> 



2we{X) 



8fg{X)\f 8fe{X) 
\ 89 



g=g° 



whe 



Eo,i = (27t)-2E 



V 961 

l^ 8[f^wg - 2YfgWg + {Y^ ~ (7|^)u.e] 



g=go 



(u) 



Pl{u) 



■ Au 



89 



g=g° 



p*{u) 



du 



The resulting rate when (fgwg) and {fgWg), as well as their derivatives with respect to 9 up to order 3 
satisfy (Ri), are given in the following corollary. 

Corollary 6.1. Under the assumptions of Theorem 6.1, assume that p;, satisfies (N2) and that for all 9 Cz 0, 
ifewg) and (fgWg) and their derivatives with respect to 9 up to order 3, satisfy (Ri). Then Corollary 3.1 
still holds with wfg replaced by wgfg. 

6.2. Construction and study of the risk of the estimator 92 

In the same spirit as 9i , we propose another estimator whose construction is based on sufficient conditions 
allowing to construct a direct and y^-consistent estimator of SgO g(9) defined in (6.1), using the observations 
(Yi,Zi), i = l,...,n. 

We say that the conditions (Cii)-(Ci4) hold if there exists a weight function wg and there exist three 



functions 
(Cii) 



£ j, J = 1, . . . , 3, not depending on g such that for all G and for all g 

Wg{x)g{x)dx = / <Pg^e,3{^)h{^)<iz, 



yfg{x)wg{x)fY,x{y,x) dydx = J y<Pg^E:2{z)fY,z{y, z) dydz 
and wg{x)fg{x)g{x)dx = <Pg^^^i{z)h{z) dz; 
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(C12) For J - l,2,3,E[supege l^%^(^)|] < 00; 
(C13) For j:= 1,2,3, and for all 61 e6',E[|^%-i(Z)|2]< 00; 
(C14) For j = 1,2,3, and for all 6> £ 0M[\ ^'ggV'^ {Z)\] < 00. 
Under (Cii)-(Ci4), we propose to estimate Sgo_g{9) by 

1 " 

■^"■^W = -^[(^.' - <^l2)^e^sAZ^) ' 2Y,^e^ea{Z^)+^e,e^{Z^)]. (6.4) 
1=1 

Using this empirical criterion we propose to estimate 9^ by 

6*2 = arg min Sn,2 {Q) ■ (6.5) 



We refer to Section 5.2 for details on how to construct such functions 'Pe,e.j, J = 1, • ■ • ,3. 



Remark 6.2. As in the first estimation procedure (see Remark b.l), the main difficulty for finding such 
function ^e,e,j ^^es ii^ the constraint that we expect that they do not depend on the unknown density g. If we 
relax this constraint, obviously there are a lot of solutions. 



6.3. Asymptotic properties of the estimator 62 



Theorem 6.3. Consider the model (1.1) under the assumptions (Ai)-(A5), (I2i)-(I23), (Ni) and the con- 
ditions (Cii)-(Ci4). Then 62, defined by (6.5) is a ^/n- consistent estimator of 6'^ which satisfies moreover 
that \pn(Q2 — 0^) A/'(0, S2), with E2 that equals 



E 



2weiX) 



dfe{X)\(dfe{X) 
dB 



e=e'> 



So,2 E 



2we{X) 



dfg{X)\fdfe{X)y 



de J\ 89 



e=e° 



where £0,2 equals 

'di^e.eAZ) - 2Y<Pe.eAZ) + {Y^ - a| 2)^'eo,e.3(^)) 



E 



961 



e=e° 



/d(<Pg^,AZ) - 2Y<Pe,eAZ) + {Y^ - Cjl2)^eo,eAZ)) 



86 



e=e'> 



Remark 6.3. Also note that by denoting 'I'e,e,i = (wefg)* /p^, 'l'e.e,2 = {wefe)* /p% and <Pe.s,3 — Wg/p^, we 
get that Sg.i = Eo,2 with Eq^i defined in Theorem 6.2. 



Remark 6.4. The same comparison between (Cii)-(Ci4) and (C8)~(Cio) holds with w replaced by wg and 
with the (I>e,£,j defined by 1 = {wefg)*/p*, ^e_e.2 = (wefe)* /pI and „ 3 = {fewg)*/p* (See Section 5.2). 



7. Examples and methodological advice (2) 



Example 8 (Growth curves 1). Consider the Model (1.1) with known cr^ 2 ^ /e (2;) = 6*1/(1 + 6*2 exp(6'3a;)) 
andp^ satisfying (N2) withp<2. Let K* [t) = l|t|<i and let we{x) = (1 + 6*2 exp(6'3x))^ exp{— x^/(4/3)}. Then 
the conditions (C8)-(Cio) as well as conditions (Cii)-(Ci4) are satisfied. Consequently, the estimators 9i 
and 62 are ^/n-consistent and asymptotically Gaussian estimators of 9'^ , with the same asymptotic variance. 
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Example 9 (Growth curve 2). Consider the Model (1.1) with known (t|2, fe{x) =62 + {Oi — 02)/{^ + 
exp(03 + 04a;)) and Pe satisfying (N2) with p <2. Let K he such K* (t) = l\t\<:i and Ze£ we(a;) = (1 + exp(03 + 
04a;))^exp{— x^/(4/3)}. Then the conditions (C8)-(Cio) as well as conditions (Cii)-(Ci4) are satisfied. Con- 
sequently, the estimators di and 82 are \/n- consistent and asymptotically Caussian estimators of 9^ , with 
the same asymptotic variance. 

Example 10 (Cauchy regression function 2). Consider the model (1.1) with known a'^2' fei^) = 
1/(1 + 9x^) and p^ satisfying (N2) with p <2. Let K be such K*{t) = l|t|<i and let wo{x) = (1 + 
0x^)^exp{— x^/(4/3)}. Then the conditions (C8)-(Cio) as well as conditions (Cii)-(Ci4) are satisfied. Con- 
sequently, the estimators 0i and O2 are y/n- consistent and asymptotically Gaussian estimators of 9'^ , with 
the same asymptotic variance. 

In these three last examples, we see the importance of the weight function we, with the improvement of 
the rate of convergence by taking we depending on 9. Clearly, for such regression function, the estimator in 
[29] does not achieve the parametric rate of convergence whereas, 9i or 02 do. 

8. Proofs of theorems 

We only detail the proof of Theorems 6.1, 6.2 and 6.3 as well as Corollary 6.1. The proofs of Theorems 
3.1, 3.2, and 5.1 and Corollary 3.1 follow the same lines with we replaced by w and by putting a^^2 = in 
the proofs (and only in the proofs). In the sequel, we denote by C an absolute constant whose value may 
vary from one line to the other, and we always mention the dependency of a constant C with respect to 
parameters. For instance C{a, b) stands for a constant depending on a and b. 

8.1. Proof of (1) of Theorem 6.1 

The main point of the proof consists in showing that for any 9 in O, Wy{Sn,i{9) — Seo^g{9))'^] = o(l), with 
Se^\g{9) admitting a unique minimum in 9 = 0". The second part of the proof consists in studying uj2(n,p) 
defined as 

uj2{n,p)^sup{\S,-,,i{9)-Snsn \\9-9'\\,,<p}. 

By using the regularity assumptions on the regression function /, we state that there exist two sequences 
Pk and efc tending to 0, such that for all fc G N 

lim P[c^2(",Pfe)>efc]=0 and that E[iuj2in, pk)f] ^ 0{pl). (8.1) 

n — *oo 

Let us start with the proof of 

E[{SnA0) ~ Seo,g{9)f]=o{l), for any and as n^cx). (8.2) 
We have to check successively that for any 9 £ 0, 

E[5„a(6')] -S'eo,g(6l) =0(1) and Var(S'„,i(6i)) o(l), asn->oo. (8.3) 

For both the bias and the variance, we give two upper bounds, based on the two following applications 
of the Holder's inequality 

|('Pl,</'2)| < ||^l||2||'^2||2 and |(</5i,(p2)| < ||¥'i||oo||l^2||l- (8.4) 

Proof of the first part of (8.3). According to Lemma A. 2, we have 
E[5„a(e)] = - fefwe - a| 2^9) * A"„,c„(^)] 

= E[(r2 _ al^)wei^KcSX)] - 2E[y (/eu;e) * ifc„(^)] + E[(/e'«ie) * /^c„ (^)], 
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and hence 

nSnAe)]-Seo^g{e) = {2nr\{f^„gr,w;{K*c^{-) - 1)) - n-^ifgog)* ,{fewgnK*c^ - 1)) 

+ {2n)-\g*,{f^wenKh„-l))- 

Consequently, under the assumption (A3), |E[S'„_i(0)] — Sgo g{6)\'^ = o(l). Indeed, according to (8.4), a first 
upper bound of |E[S'„^i(0)] — Sgo_g{9)\ is given by 

{2nr^{fl,grUw;{K*c^ - + n-^\\{fgogrU{fgwgr{K*c^ - 

+ (27r)-i||/||2||(/,W(/^£„-l)||2 

and hence 

\E[Snj{e)]-Sgo,g{e)f 

< C{fgo,Wg)[\\w*g{K*c^ - l)\\l + \\{wgfgnK*c^ - + || (u'e/e')* (Ac„ - l)ll2]- (8-5) 

And according to the second part of (8.4) we also have that |E[S'„_i (6*)] — 5^0 ^(6*)! is bounded by 

{2n)-'\\{f^ogr\\J\w;{K*a^-l)\\,+n-^{fgagr\^^^^^^ 
+ i2n)-'\\g*\U\if^wgnK*c^-l)\\, 

and hence 

\E[Sr,,m]-Sgoje)f 

< C{fgo,Wg)[\\w;{K*c^ - 1)11? + \\{wgfgnK*c^ - 1)||? + || (u^e/e' )* (A'£„ - (8.6) 

Consequently, by combining the two bounds (8.5) and (8.6), wc get that |E[S'„ 1(6')] — Sga g{0)\'^ is bounded 

by 

C{fgo,wg)mm{\\w;{K*c^ - + ||(^e/e)*(A-£„ - I)!!' + WiwefinK*^^ - 

w^UKh^ - ml + \\i^'efonK*c,^ - ml + uwef^nKh^ - mil} 

that is by applying Lemma A.l, 

\E[Sr.,m] - SgoJe)f <C{fgo,WgAr)C-^^+^'-^^ + ^'^^^^ 

and the first part of (8.3) follows. □ 

Proof of the second part of (8.3). Since the variables are i.i.d. random variables, the stochastic term 
on the left-hand side in (8.3) has variance 

Var[5„4(0)] < n-'E[\{{Y - Jgfwg ~ al^wg)^ Kn^cSZ)\'] 

< 3n-i{E[(r4 + ayiwg^K^^cSZ)^ + AE[Y^\{fgWg)*R\cAZ)f] 
+ E[\if^wg)^Kn^cjZ)\'']} 

< (^^^ {E[f^oiX){wg*Kn,cyiZ)] + (E(e^) + 42)E[(i«e * if„,cj'(^)] 

+ E[f^oiX){{fewe)*K„,cAZ)f]+E[{{f^wg)*Kn,cjZ)f]}, 
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that is. according to Lemma A. 2 

Var[^„4(0)] < (^^^{(((/4„.g)*p,),(w;e*i^„,cJ') + (E(e") + 42)((.9*Pe),(^«e*^r.cJ') 

+ {{g*Pe)difeWe)*K„.cJ)}- 
On one hand, according to (8.4), under the assumption (A3), 

Var[5„,i(0)] < (^^){[||(/,^05)*Pe|loo + (E(e')+42)ll.9*Pe|lco]|kfl*i^n,cJl^ 
+ [\\{f0«9)*Pe\\oc~+'^'i,2\\9*Pe\\oo]\\{feW0)*Kn^cjl 

+ II .9 * 1 1 00 1 1 ifewe ) * A'„^c„ II 2 } ' 



and we get that 

Var[S'„4(6')] < C(cr| 2, 0-5,4, /e, 'We,^^)^"^ 



7^* 2 



{fewer 



K* 2 



7.--* 



(8.7) 



On the other hand, again applying (8.4) under the assumption (A3), we have 



C 



Var[5„,i(0)]<-{[||(/4o.g)^p,||^ + (E(e^) + 42)ll5*ft||i]||i«fl*/^ru 

+ [ll(/eo5)*P£|li + cr|,2ll5*Pe||i]||(/ewe)*X„,c,. 
+ WfJ * PeWiUfeWe) * K,,^cJ\l^}, 
and hence we get that 



c„iiL 



Ya,-r[Sn,iiO)] < C((t| 2, 05,4, /e, '!«e,P£)n ^ 



{weY 



K* 



{fewer 



K, 



K* 

{fewe) — - 



(8.8) 



By combining (8.7) and (8.8) and by applying Lemma A.l in the Appendix, we get that under assumption 
(A3), Nw{Sn.\{&)\ is bounded by 



n ^C(cr|, 0-5,4, /e, We, Pe) X min 



{wer 



{feweY 



K* 



{feWeT 



{wer 



K* 



Pe 



{fewer 



K*r 



{f^weY 



.K*r 



In other words, 



Var[^„,i(0)] < C(a|, ^5,4, fe, we,Pe) max[l, 2a+(i-p)+(i-p)- 



-2bC;+2/3CP 



\/n, 



and under (3.7), then (8.2) is proved. 

It remains now to check that there exists two sequences pk and ek tending to 0, such that (8.1) holds. 
First write that 



|^„,lW-^rM(^')l 



n 1 

J2Mfe'We' ~ fewe)]*K,,^cAZi)- -Y.[{fe'We' ~ feWe)]*K,,,cAZ^) 
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961 

where \\e\\p- < \\e-9'\\e2. It follows that for ||6'-6''||f2 < pk, 

f2„,, I /v2 „2 



|^„,lW-5„,i(0')l< — 



^fd[Yifewg + f^we + {Y^ - ul^)we] 



00 



Hence (8.1) holds since for C„ satisfying (3.7), by using the same arguments as for the proof of (8.2), we 
have that for all G 6* 



E 



1=1 \ 



89 



0(1), 



and hence for all fc e N, E[sup||0,_0||^,<p^ \Sn,\{9) — Sn,\{&y\\ = 0{pk) as n — > oo. 
8.2. Proofs of (2) of Theorem 6.1 and Corollary 6.1 



□ 



If we denote by sl^\ and 5"^^^ the first and second derivatives of Sn,i{9) with respect to 9, by using classical 
Taylor expansion based on the smoothness properties of i— *■ wgfg and the consistency of 9i, we obtain that 

= sl^lid^) = 5« (00) + s';^li9'm 0') + RnAOi 0"), 

with Rn.i defined by 
»i 



Rn,l 



[4^1(0" + .(^i-O)-^i'kOd.. 



This implies that 

^i-0O = -[5S(^^") + i?„,i]"'5i^l(O- 
Consequently, we have to check the four following points. 



(8.9) 



(8.10) 



(i) E[(5i;{(0O) - 5^;y0°))(5i:}(e°) - 5^iy0°))^] = o[v,,^i]. 

(ii) n\s'^l{0')-Sf^^^{9^)\\l] = o{l). 

(iii) Rn,i defined in (8.9) satisfies E(|ji?„^i jj^a) =o(l) as n^oo. 



(iv) noi-oX^<^n{s^nl{e')Y[{sf^^^{9^))-'^^ 

The rate of convergence of 9i is thus given by the order of S^^l{9°) - 5^J'g(6lO) = S','^^}(6'"). 
Proof of (i). Write that 



■E 

i=l 



d[m~fgf^Cjl^)wg] 



89 



8[{{Y-fgiX))-'-al,)woiX)] 



89 



412 C. Butucea and M.-L. Taupin 

Study of the bias. According to Lemma A. 2. E[S'^"'^|(^°)] is equal to 



-2E 



fe<X) 



*Kc„{X} — (A) 



-E 



961 



-E 



89 



86 



that is 



E[4!i(^")] = -2((/.o,r,(^(^ 



6=8" 



+ ((/|ogr, 



/ 8we 
[ 89 



e=e° 



0=8° 



(^c„ - 1)) 



On one hand, according to (8.4), mrdcr the assumption (A3), the bias is bounded in the foUowing way 



E 



8Sn,m 

89, 



0=8" 



<n-'\\ifgogr\\, 
+ (27r)"i|l(/,2„g) 



/ 8if0Wg) 



0=8« 



V 99, 



f 8w0 



8=0° 



0=8° 



and consequently, 

'8SnAS) 



E 



89, 



8=00 



< C{fgo,Wgo,Pe) 

( 8{wgfe) 



/ d{wg) 

V d9, 



ii<L - 1) 

(Kh^ - 1) 



V 99, 



0=8° 



(8.11) 



8=0° 



(Kh^ 1) 



And, on the other hand, the bias can also be bounded in the following way 



E 



8Sn.iie) 

89, 



0=8° 



<n-'\\ife°gr\i 
+ (27t)-i|1.9ll 
+ (27r)-i||(/,2„g)*|| 



/ 8{f8W0) 



V 99, 
diPgWe) 



0=8° 



89, 



dwe 



8=0" 



0=8° 



and consequently, 
'8S,,s{0) 



E 



89, 



8=00 



f 8{wgje) 



V d9. 



0=00 



< C{fgo,Weo,Pe) 



/ 8{wg) 

V d9. 



+ 


(d{wefl) 


1 


V 99, 



8=00 



8=00 



(KL - 1) 



(8.12) 
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By combining (8.11) and (8.12), we get that 



E 



e=e° 



[1] (aO\ n[2] (aO\ 



with B™A^^), q = 1,2 defined in Theorem 6.1. 

Consequently, by applying Lemma A.l, we obtain that 



' (dSn.m 






e=e°) 



< cP{feo,wgo,pe,b,r)C, 



-2a+(l-r) + (l-r)_g-2hC; 



Study of the variance. For the variance term, it is easy to see that 



Var 



55„,i(e) 



90, 



e=e° 



< — E 
n 



d[-2YJgwg + f^wg + (Y^ - al^)we] 



90, 



1 2 



that is, according to Lemma A. 2, 'Vai-{dSn,i{0)/99j\g^go) equals 



C + o(l) 



( d{fgWg) 



0=8° 



A 99, 

+ ^((/eo + (7^,4 + 4/|o(t|,2 - 4,2 + 4/90^^^,3)5) *Pe, 
'f9{f^Wg) 



9wg 



e=e'> 



'VV 90, 



0=9° 



* Kn.C 



It follows that according to (8.4), Ya,r{dSn.i{0) /d0j\g^go) is less than 



C 

-\ ['^l2\\9*Pe\\^ + \\if0og)*Pe 



f 9{fgWg) 



V 96, 



0=00 



+ !1 ((/eo + a^,4 + 4/2„a|,2 - ^2 + 4/eoaj,3)5) *Pe\\, 
'9{PgWg) 



dwe 



9=00 



that is 



Var 



\9*Pe 



9Sn,l{0) 



80, 



d0. 



< 



} = 0O 



-kK, 



n,C„ 



0=9° 



Ci<7l2,.f0oJgo]j,Wgo,p^) 



( 9{we) 
V 90, 



00/ Pe 



f 9{f0Wg ) 




2 

+ 


f9{f^Wg) 




2-1 


V 90, 


0=goj Pe 


2 


\ 90, 


g=0oj P*e 


2- 



(8.13) 



And once again, according to (8.4), another bound the variance term can be obtained to get that 
Ya,T:{d Sn,i{0) / 90 j\9=0o) is bounded by 



C_ 

n 



Wl2\\9*PA\l + \\{f909)*Pe\\i 
(9{PgWg) 



( 9{f0Wg) 



\9*Pe\\l 



V 90, 



= 00 



V 90, 
2 



0=00 



n,C„ 
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+ II ((/eo + a^A + 4/|oa| 2 - ^2 + 4/90^^,3)5) *Psll 1 



/ dwe 














00 J 



that is 



Var 



e=e° 



< 



C{crl2,.feoJgl]j,weo,Pe) 



00/ Pe 



(8.14) 



/ difewe) 




2 

+ 






2-1 




0=90/ Pi 


1 




9=0«J Pi 


1- 



By combining (8.13) and (8.14), wc get that 



Var 



with v}^], g = 1, 2 defined in Theorem 6.1. By applying Lemma A.l, we get that 



a^n,l(g) 



0=0° 



Cicr'l2j0o,f^l\,W0o,pg) 



< — min{yW(0°),yi^|(0°)}, 



Var 



0=00 



< 



Cv2 {(^l2Jeofgl]j,Wgo,p^) 



c[l,C2"-2a+(i-rt+(i-rt-cxp{-26C; + 2/3C^}]. 



The rate of convergence of 9i denoting by ifin'fiJi corresponds to the best choice for the sequence C* , mini- 
mizing the sum of the variance YaT[sl^\{e°)] and the square of the bias (E[S'i^J(6'0)] - S'^^He°)){E[s';^l{0°)] - 

5W(0O))T = (E[5«(0O)])(E[5«(0O)])T. 

Polynomial noise (see (N2) with /? = p = 0). 

• If for j = 1, . . . , m, d{f0W0)/d9j\g^go, dwg / d9j\s^go and d{fgWg)/d9j\g^go satisfy (Ri) with r = 0, then 



E 



dSrM 



0=0° 



<CMo,wgo,b,al2)C-^''+\ 



and 



Var 



0=0° 



) < a,(E(/2o(X)), al2,/g^,\f0o,W0o,p,) max[l, C^f-^a+ij/^ 



(8.15) 



It follows that if r = and a < a + 1/2 then 

and ^ 0(n(i-2-)A2")). 
If r = and a > a + 1/2, then 

C: = ni/(2a-i) and ipl^O{n-^). 
• If for j = 1, ... , m, d{fgWg)/d9j\g^go, dwg/d6j\g^go and d{fgWg)/d9j\g^go satisfy (Ri) with r > 0, then 

"55n,l(0) 



E 



d9i 



0=0° 



< a.(/eo,42,/io\u;eo)C-2-+(i-'-)+(i-'-)- exp{-26C;} 



and 



Var 



d9i 



0=00 



Cv2 if 00 , f^|,2 ' fgo^ ' ^0° ' Pe) 



< 
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It follows that 

"logn -2a+ (1 - r) + (1 - r)_ 



c: 



2b 



2br 



log 



logn 
~2b' 



l/r 



and ipf-^ = 0(n ^) 



(8.16) 



Exponential noise (see (N2) with p > 0). 

• If for j = 1, . . . , m, d{fgWg)/d9j\g^go, dwg/d9j\g=go and d{fgWg)/d9j\g^go satisfy (Ri) with r = 0, then 



E 



and 



Var 



<C6,(/eo,42,/io\«;eo)C-2-+l, 



] < C,Moo,crl,,f^l\wgo,p,)Cl'^-'^+('-P)+i^-'') 



exp{2/3C^} 



It follows that 

logn 2a+(l — p)- 



c: = 



2/3 



2p/3 



log 



logn 



1/p 



and i^^j = O 



logn 



(l-2a)/p- 



(8.17) 



• If for j = 1, . . . ,TO, d{fg'wg)/d6j\g^go, dwg/d9j\g^go and d{fgWg)/d9j\g^go satisfy (Ri) with r > and 
{r > p} or {r = p and b> fi} or {r = p, 6 = /3 and a > a + 1/2} then 



E 



a5„,i(^) 



< a. (E(/^c(X)), a|^2, L(/^o'^«;eo), L(/(,o/^o'«^eo))C- 



-2a+(l-r) + (l-r)_ 



exp{-26C;}, 



and 



Var 



55„,i(0) 



< a, (E(/^o (X) ) , , Li/gl^WgO ),L{fg. f';,>WgO )) / U 



It follows that 



logn — 2a + (1 — r) + (1 — r)_ /logn 



26 



2br 



log 



26 



-1 l/r 



and (p'^ = 0{n 



(8.18) 



• If for j = 1, ... ,rn, d{fgWg)/d9j\g^go, dwg/d9j\g^go and d{fgWg)/d9j\g^go satisfy (Ri) with r>0 and 
r < p or r = p and 6 < /3 then 



E 



dS„A9) 



89. 



e=e° 



<a(E(/,2o(X)),a22,L(4o^^u;eo),L(/,o4o^'«;,o))C-- 



and 



Var 



55„,i(0) 



9=61" 



< a,(E(/2o(X)), ol^Mf^^e^). Lifgof^gl^Wgo)) 

X C^"-2a+(i-p)+(i-rt_g^p|_2&c';; + 2/?C^}/n. 



It follows that the bias is by a logarithmic factor larger than the variance and 



c: = 



logn 2a+ (1 - p)- - {1 -r)- 



log 



logn 



1/p 
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and 



(logn)( 



-2a+(l-r) + (l-r)_)/p 



exp< —26 



logn 



r/p 



li r = p, b = (3 and a < a + 1/2 then 



Var 



V 96. 



0=9° 



a 



2Q-2a+l 



The resulting rate is thus ipl = 0[(logn)(2"~2a+i)/r/^j^ 
Proof of (ii). By using that 



□ 



e=e° 



■K^^cSZ^). (8.19) 



we write 5^'^ (0°) - 5^o^g(^°) = Ao + Ai + A2 + ^3 with 



{X) 



1 " 

-E 



52 



^3 = iE(^/-- 



d0,d9k 



{X) 



e=e'> 



do J 86 k 



(X) 



As soon as wgfe and wgfg and their derivatives up to order 2, satisfy (Ri), then for C„ satisfying that (3.7), 
we get that Aq = o(l) and Aj = Op(l) as n ^ 00 for j = 1, . . . , 3, and (ii) is proved. □ 

Proof of (iii). Again using (8.19), the smoothness properties of the derivatives of wgfe and wefg up to 
order 3 and the consistency of ^i, we get that H-Rn.ilU^ = Op(l) as n ^ 00. □ 

Proof of (iv). Let us introduce the random event E„ = C\j ^ En.j^k, where 



such that 



d^SgoJO) 



00 j d0k 

According to (8.10) and (8.9), we have 



9=9° 



E\\0i - 0°\\% = E[\\9i - 9"\\p1eJ + n\\0i - ^"llf^lficj 

< E[\\0i - 0"||'.l£;J + 2 sup \\0\\%PgajE^^) 



q0||2 



,0|i2 



9=9° 



^ld^Sgo^g{9) 



2 ae^dOk 



9=9° 



9£0 



<E[(5«(0"))^[(5i^i(0°)+i?„a)"y(^S(^°)+^«.i)"'^S(^°)ii^j 

+ 2snM%^9oAK) 

9G0 



<C4^™ sup 



d'SgoJ0) 




d0jd0k 


9=9° 



IE[(<k^°))^^S(^")] + 2sup||0|||.P«o,,(i?;^) 

see 
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<Cr™ sup 

j.k 









e=e° 



^2+2sup||0|||.Peo,,(i?O. 



It remains thus to show that ¥'0a g[Ef^) ~ o((/3^) with 



supE 



By Markov's inequahty, for a p > 2, 



61=00 



< 



3 = 1 k=l 



with 



89, 89 



9=60 



+ {Rns)j,k 



< 2P'^ 



d'SgoJ9) 



89, 89k 



-E 



89i 89k 



e=e° 



2P"^E 



E 



89, 89k 



89, 89k 



0=9° 



+ {RnAj.k 



In other words, 
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e«. 



d^SgoJ9) 



89, 89k 



E 



,2 



-E 



89, 89k 



e=e° 



39, 89k 



9=00 



8^S„AS) 



89, d9k 



61=00 



Now we apply the Rosenthal's inequality to the sum of centered variables 



i=l 







\(8^Snj{^) 


0=00 )- 


V d9jd9k 


= 00/ 


[\ d9-d9k 





where Wn,ij,k equals 

'd^[-2YJewg + pgwe + [Y^ - cTl^)we] 



-2E 



89 J d9k 

8-'[-2Y,Jewe + PgWg + [Y^ - al^)wg] 



89 J 89k 



+ 22p-2E|(i?„akfcr 



Lemma 8.1 (Rosenthal's inequality [27]). For Ui,. . . , Un, be n independent centered random variables, 
there exists a constant C (p) such that for p > 1 , 



E 



<C{p) 



VE[|c/,n+ VE[[/f 



p/2- 



(8.20) 
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Consequently, 



E 



00 J dOk 
Take p = 4 to get that 



-E 



E 



80 j d0k 



-E 



30 J dOk 



e=e° 



PI 



00, OOk 



4i 



<C(4)[n-3E|W„,i|4 + n-2E2|W„,i|2 



Therefore, under the eonditions ensuring that (E[5go'' {0^) — S^^\{0^)\^)j,k =o(l), we have 



Now, by using the definition of Rn.i combined with (8.19) and the smoothness properties of the derivatives 
of {wefe) and [wefg) up to order 3, we get that E((i?„^i)|j.) =o(||6'i — 6'''|j^2), and we conclude that 



+ o{^l) + om\Oi-eX.]). 

8.3. Proof of Theorem 6.2 



□ 



The proof of Theorem 6.2 follows from Theorem 6.1 from the conditions (C8)-(Cio) and from the central 
limit theorem with the Lindeberg condition (see, for instance, [4]). The main point of the proof lies in the 
proof of 



V^5i^l(0°) ^ AA(0,So.i), 
where So.i is defined in Theorem 6.2. We start by writing that 
V^S^^l{0')^VT^[S^l{0'^)-S^^l{0')] 

Vn 



(8.21) 



^^+v^[E(5W(0O))-4j;^(^?")]. 



where 



Vn, 



0[{{Y,~fef~cjl^)wo] 



00 



0[m^fer^al,)we] 



00 



e=do 



The proof of (8.21) follows from the three points 
(A-1) n-iEr=iK.KT. ^ So,i; 

(A-2) For all e>0, n-^Zt^Vn^^V^.^y^^^^^^^^^^^J; 
(A-3) V^[E{si'\{0^)) - Sil\{0'>)] ^ 0. 

Proof of (A-1). The conditions (C8)-(Cio) ensure that Vnj =0(1) for j ~ 1, . . . ,d. Hence, (A-1) is 
checked. □ 
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Proof of (A-2). It suffices to prove that for all e > and for all j = 1, . . . , d, 



E 



HVn.i.j\>iiVn 



i=l 



^o(l), 



where Vni j is the jth coordinate of the vector Vn Now (A-2) is checked by writing that under (A5) and 
(CsHCioj, 



n 



2—1 ^ i — 1 



<^xO(l). □ 

Proof of (A-3). Conditions (C8)-(Cio) ensure that for all €„ the variance term Vnj = 0(1) for j = 1, . . . , d. 
Hence, it is possible to choose C„ such that (A-3) is checked. □ 

8.4- Proof of Theorem 6.3 

As for the proof of Theorem 6.1, the main point of the proof consists in showing that for any 9 € &, 
^[{Sn,2{0) — Seo g{9j)'^] ~ 0(1), with S0o^g{9) admitting a unique minimum in 9 = 9^. The second part of the 
proof consists in studying uji{n,p) defined as uji{n,p) = sup{|S'„^2(^') — Sn,2{&')\' \\d — 9'\\2 < p}- By using 
the regularity assumptions on the regression function /, we state that there exists two sequences pk and efc 
tending to 0, such that for all k gN, 

lim P[cji(n,pfe)>efc]=0 and that E[{u;i{n, pk)f] ^ 0{pl). (8.22) 

n — >oo 

Under the conditions (Cii)-(Ci3) and by applying the law of large numbers, we get that for any 9 G 
6», mSnM^) - 5eo,g(6l))2] = 0(1) as n ^ 00 and E[(5„,2(6') - S'„,2(6''))^] = o(l) and consequently, ?2 is a 
consistent estimator of 6*°. By using classical Taylor expansion based on the smoothness properties of the 
regression function, with respect to 9 and the consistency of ^2, we get that ~ S^^li^^) = ^nli^'^) + 
Sl^.l{d°)(02 - 0") + RnM^2 - 0°), with i?„.2 defined by 

Rn,2^ [\sl'},i9"+si9,~9"))9-S'^'},i9")]ds. (8.23) 
Jo 

This implies that 02 — = ^[Snli^^) + Rn.2]^^ sl^\{9^) . The y^-consistency and the asymptotic nor- 
mality follow by applying the central limit theorem with the Lindeberg condition to sl^\{9^) to get that 
V^SI,]1{9°) ^ AA(0,So,2). 

Appendix: Technical lemmas 

Lemma A.l. Let ip a function such that ip belongs to L2(IR) satisfying (Ri). Then 

\ip*iu)\du<-^^C~'^+'-^cxp{-bC:}. (Al) 
|u|>c„ R{a,b,r} 

Furthermore, if Pe satisfies (N2), then 



l\u\<Cr. 



< ur » ^^"^l ^r^f ^ "iax[l, C^-^+i"'' exp{-6C; + /JC^]. 



\pl{u)\ R{a,P,p,a,b,r)C{pe) 
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Lemma A. 2. Let (f such that E{\(p{Y,X)\) is finite and let <P such that E{\<l>(U)\) is finite. Then 
and 

E[v.(y,X)<?*if„,C„(t/)]' = j {{^{x,-)fY.x{x,-))*fe,{<^*Kn,cy)dx. 

Proof. If we denote by fy.z.x the joint distribution of {Y, Z, X), then 

fY,z,x{y,z,x) = fY,x{y,x)fe{z- x) and fY,z{y, z) ^ fY,x{y, ■) * fe{z)- 
By construction Kn,Cn *Pe — ^c„j a-nd hence Parseval's formula 

E[ipiY,X)^*Kr,,cAZ)] = j j ^iy,x)fY.xiy,x) J Kr,,cjz)fe{z ~ x)dzdydx 

^{2nr^ j j ^{y,x)fy,x{y.x) j ^*{u)^^p^f:{u)e-'-^ dudydx 



'Piy,x)fY,x{y,x) J ^{z)Kc^{x - z)dzdydx. 
The second equahty follows by writing that 

E[ip{Y,X)^*K„^cAZ)f = III ip{y,x)i$*K„^cAz)ffY,x{y,x)fe{z-x)dydzdx 

ip{y,x){^-kKn,c„{z)ffY,x{y,x)fe{z ~ x) dydz dx 

= J ((^(y,.)/y,x(y,-))*/e,(<^*if„,cJ')d2/. □ 
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